Sorry, last minute "fix" slipped in and there is a one unwanted reference to bootstrap tools that fails the stdenv build. The fix is one-liner and will not rebuild too much if you already tried the branch:

--- a/pkgs/stdenv/linux/default.nix
+++ b/pkgs/stdenv/linux/default.nix
@@ -459,7 +459,8 @@ in
       # other purposes (binutils and top-level pkgs) too.
       inherit (prevStage)
           bash gettext gnum4 bison
-          perl texinfo which xz
+          perl texinfo which
+          # xz - rebuild to pull in rebuilt version into final stdenv
           zlib linuxHeaders libidn2 libunistring;

       ${localSystem.libc} = getLibc prevStage;

I'll push it in a few minutes along with comment updates.

I'll push it in a few minutes along with comment updates.

Pushed the update. Was able to build stdenv on x86_64 and a bunch of c++ packages on top of it.

Should be ready to test.

Currently attempting to bootstrap my entire server from this branch, will see if it builds.

I'm rebuilding my entire aarch64 test VM with a pretty standard Xfce + Firefox on this branch. Then I will try again with GCC 11.

I've successfully built about two thirds of my server config with this branch, including pretty much every compiler under the sun (Clang, GHC, gfortran, rustc, Ruby, Python, Erlang, Elixir) in the chain. I'll leave it running to build the server config and also my desktop config overnight and see how that goes, but it's very likely this is good to go on x86_64-linux.

Aaaand the server config is built, with one unrelated failure (#209118). Time for the desktop, and time for me to go get some sleep.

The desktop build ended up running out of disk space, but it got almost all the way there. I think this should really be good to go. Has anyone tested aarch64 with GCC 11 yet?

Does this still not violate the stance that none of the files in bootstrap or generated by bootstrap tools are not used directly?

Got another snag: vte (uses clangStdenv) failed for inability to dins libstdc++ librray. It's a side-effect of --sysroot= hack. Added a safe workaround for it to provide cross-looking gcc.lib paths with:

--- a/pkgs/development/compilers/gcc/11/default.nix
+++ b/pkgs/development/compilers/gcc/11/default.nix
@@ -221,6 +221,7 @@ stdenv.mkDerivation ({
   };

   targetConfig = if targetPlatform != hostPlatform then targetPlatform.config else null;
+  targetPlatformConfig = targetPlatform.config;

   buildFlags = optional
     (targetPlatform == hostPlatform && hostPlatform == buildPlatform)
diff --git a/pkgs/development/compilers/gcc/builder.sh b/pkgs/development/compilers/gcc/builder.sh
index 113bd83ea53..71a997a7df9 100644
--- a/pkgs/development/compilers/gcc/builder.sh
+++ b/pkgs/development/compilers/gcc/builder.sh
@@ -203,6 +203,17 @@ preInstall() {
         ln -s lib "$out/${targetConfig}/lib32"
         ln -s lib "${!outputLib}/${targetConfig}/lib32"
     fi
+
+    # cc-wrappers uses --sysroot=/nix/dir/does/not/exist as a way to
+    # drop default sysheaders search path. Unfortunately that switches
+    # clang++ into searching libraries in gcc in cross-compiler paths:
+    #   from ${!outputLib}/lib (native)
+    #   to ${!outputLib}/${targetPlatformConfig}/lib
+    # We create the symlink to make both native and cross paths
+    # available even if the toolchain is not the cross-compiler.
+    if [ ! -e ${!outputLib}/${targetPlatformConfig} ] ; then
+        ln -s . ${!outputLib}/${targetPlatformConfig}
+    fi
 }

Applied to all gcc versions. It fixes vte and other clang-based builds. Did not seem to break pkgsLLVM or cross-builds either.

Restores excessive rebuilds of xz, texinfo and friends to make main commit leaner. Will reintroduce it separately this evening.

Restores excessive rebuilds of xz, texinfo and friends to make main commit leaner. Will reintroduce it separately this evening.

Proposed 2 of 3 perl builds removal separately in #210118

Proposed one texinfo removal from binutils as #210132

Filed upstream bug for poor build parallelism in gettext: https://savannah.gnu.org/bugs/index.php?63641

No functional change. No need to retest.

Rebased against current staging. This left us with a single gcc stage addition change (if we ignore #210741)

While at it added missing minor rename:

--- a/pkgs/stdenv/linux/default.nix
+++ b/pkgs/stdenv/linux/default.nix
@@ -415,7 +415,7 @@ in
       gcc-unwrapped =
         let makeStaticLibrariesAndMark = pkg:
               lib.makeOverridable (pkg.override { stdenv = self.makeStaticLibraries self.stdenv; })
-                .overrideAttrs (a: { pname = "${a.pname}-stage3"; });
+                .overrideAttrs (a: { pname = "${a.pname}-stage4"; });
         in super.gcc-unwrapped.override {
         # Link GCC statically against GMP etc.  This makes sense because
         # these builds of the libraries are only used by GCC, so it
@@ -466,7 +466,7 @@ in
       # force gmp to rebuild so we have the option of dynamically linking
       # libgmp without creating a reference path from:
       #   stage5.gcc -> stage4.coreutils -> stage3.glibc -> bootstrap
-      gmp = lib.makeOverridable (super.gmp.override { stdenv = self.stdenv; }).overrideAttrs (a: { pname = "${a.pname}-stage4"; });
+      gmp = lib.makeOverridable (super.gmp.override { stdenv = self.stdenv; }).overrideAttrs (a: { pname = "${a.pname}-stage5"; });

       # To allow users' overrides inhibit dependencies too heavy for
       # bootstrap, like guile: https://github.com/NixOS/nixpkgs/issues/181188

@trofi I guess this PR is now superseded by #210741?

@trofi I guess this PR is now superseded by #210741?

No, this PR includes that one and adds a gcc rebuild stage step.

Rebased past gettext conflict.

@ofborg build icu

@trofi should we just close this as superseded maybe?

Sure. I did not yet get the feedback from @Ericson2314 and I think it's a prerequisite. If the PR languished for a month and problem persisted for as long as I used nixpkgs it can wait for another year.

You will need another form of the #208412

Not the first time I (and others) threw away most of the effort around the bootstrap bootstrap improvements away. I can keep the patch locally indefinitely. I already have about 200 of them :)

All I meant was that #209870 achieves the same thing and seems to be a better approach. Are there issues with that change that don't occur with this one?

Good question. I don't know. It's too hard for me to reason about it as a whole. I usually diff before/after state to see the difference. It's very hard to do there.

I usually diff before/after state to see the difference. It's very hard to do there.

I initially tried factoring it into multiple PRs, but ended up getting "i don't see the point of this" feedback for the parts that were not useful separately from external-bootstrap (example). So to avoid time being wasted on those unproductive discussions I put it all under a single PR.

If you think about it, there's really no reason for people to merge isolated parts of #209870 since none of it is useful without the whole thing.

If you want to review it in pieces it isn't hard; the major changes are in only two files:

• pkgs/stdenv/linux/default.nix
• development/compilers/gcc/common/external-bootstrap.nix

Everything else is just fixing breakage (like emacs nativeComp) and threading around new parameters.

I could make the PR smaller if it were okay to break emacs, but that's not okay.